Ref-NMS: Breaking Proposal Bottlenecks in Two-Stage Referring Expression Grounding

The prevailing framework for solving referring expression grounding is based on a two-stage process: 1) detecting proposals with an object detector and 2) the referent to one of proposals. Existing solutions mostly focus step, which aims align expressions In this paper, we argue that these methods overlook obvious mismatch between roles in two stages: they generate solely detection confidence (i.e., expression-agnostic), hoping contain all right instances expression-aware). Due mismatch, current suffer from severe performance drop detected ground-truth To end, propose Ref-NMS, first method yield expression-aware at stage. Ref-NMS regards nouns as critical objects, introduces lightweight module predict score aligning each box object. These scores can guide NMS operation filter out boxes irrelevant expression, increasing recall resulting significantly improved performance. Since Ref- agnostic it be easily integrated into any state-of-the-art method. Extensive ablation studies several backbones, benchmarks, tasks consistently demonstrate superiority Ref-NMS. Codes are available at: https://github.com/ChopinSharp/ref-nms.